Systems and Methods for Planning and Navigation

ABSTRACT

The present disclosure is directed to a planning and navigation method. The planning and method includes obtaining and rendering a plurality of images. The plurality of images are segmented to demarcate a target area. A treatment plan is determined based on the target area. The navigation method includes obtaining an ultrasound image of a scan plane including the target and obtaining a fiducial image of a fiducial pattern disposed on an ultrasound device. The obtained fiducial image is corrected and a correspondence between the fiducial image and a model image is found. A position of the surgical device is transformed to model coordinates. Then the ultrasound image and a virtual image of the surgical device is displayed to allow a surgeon to navigate the surgical device to the target using the displayed ultrasound image and the virtual image. The target is treated based on the treatment plan.

BACKGROUND

1. Technical Field

The present disclosure relates to systems and method used in relation toa surgical procedure. More specifically, the present disclosure isdirected to the use of a planning system to determine a treatment planand a navigation system to effect a treatment plan for a surgicalprocedure.

2. Background of the Related Art

Electrosurgical devices have become widely used. Electrosurgery involvesthe application of thermal and/or electrical energy to cut, dissect,ablate, coagulate, cauterize, seal or otherwise treat biological tissueduring a surgical procedure. Electrosurgery is typically performed usinga handpiece including a surgical device (e.g., end effector or ablationprobe) that is adapted to transmit energy to a tissue site duringelectrosurgical procedures, a remote electrosurgical generator operableto output energy, and a cable assembly operatively connecting thesurgical device to the remote generator.

Treatment of certain diseases requires the destruction of malignanttissue growths, e.g., tumors. In the treatment of diseases such ascancer, certain types of tumor cells have been found to denature atelevated temperatures that are slightly lower than temperatures normallyinjurious to healthy cells. Known treatment methods, such ashyperthermia therapy, typically involving heating diseased cells totemperatures above 41° C. while maintaining adjacent healthy cells belowthe temperature at which irreversible cell destruction occurs. Thesemethods may involve applying electromagnetic radiation to heat, ablateand/or coagulate tissue. There are a number of different types ofelectrosurgical apparatus that can be used to perform ablationprocedures.

Minimally invasive tumor ablation procedures for cancerous or benigntumors may be performed using two dimensional (2D) preoperative computedtomography (CT) images and an “ablation zone chart” which typicallydescribes the characteristics of an ablation needle in an experimental,ex vivo tissue across a range of input parameters (power, time). Energydose (power, time) can be correlated to ablation tissue effect (volume,shape) for a specific design. It is possible to control the energy dosedelivered to tissue through microwave antenna design, for example, anantenna choke may be employed to provide a known location of microwavetransfer from device into tissue. In another example, dielectricbuffering enables a relatively constant delivery of energy from thedevice into the tissue independent of differing or varying tissueproperties.

After a user determines which ablation needle should be used to effecttreatment of a target, the user performs the treatment with ultrasoundguidance. Typically, a high level of skill is required to place asurgical device into a target identified under ultrasound. Of primaryimportance is the ability to choose the angle and entry point requiredto direct the device toward the ultrasound image plane (e.g., where thetarget is being imaged).

Ultrasound-guided intervention involves the use of real-time ultrasoundimaging (transabdominal, intraoperative, etc.) to accurately directsurgical devices to their intended target. This can be performed bypercutaneous application and/or intraoperative application. In eachcase, the ultrasound system will include a transducer that imagespatient tissue and is used to identify the target and to anticipateand/or follow the path of an instrument toward the target.

Ultrasound-guided interventions are commonly used today for needlebiopsy procedures to determine malignancy of suspicious lesions thathave been detected (breast, liver, kidney, and other soft tissues).Additionally, central-line placements are common to gain jugular accessand allow medications to be delivered. Finally, emerging uses includetumor ablation and surgical resection of organs (liver, lung, kidney,and so forth). In the case of tumor ablation, after ultrasound-guidedtargeting is achieved a biopsy-like needle may be employed to deliverenergy (RF, microwave, cryo, and so forth) with the intent to killtumor. In the case of an organ resection, intimate knowledge ofsubsurface anatomy during dissection, and display of a surgical devicein relation to this anatomy, is key to gaining successful surgicalmargin while avoiding critical structures.

In each of these cases, the ultrasound-guidance typically offers a twodimensional image plane that is captured from the distal end of apatient-applied transducer. Of critical importance to the user forsuccessful device placement is the ability to visualize and characterizethe target, to choose the instrument angle and entry point to reach thetarget, and to see the surgical device and its motion toward the target.Today, the user images the target and uses a high level of skill toselect the instrument angle and entry point. The user must then eithermove the ultrasound transducer to see the instrument path (thus losingsite of the target) or assume the path is correct until the deviceenters the image plane. Of primary importance is the ability to choosethe angle and entry point required to direct the device toward theultrasound image plane (e.g., where the target is being imaged).

SUMMARY

This description may use the phrases “in an embodiment,” “inembodiments,” “in some embodiments,” or “in other embodiments,” whichmay each refer to one or more of the same or different embodiments inaccordance with the present disclosure. For the purposes of thisdescription, a phrase in the form “A/B” means A or B. For the purposesof the description, a phrase in the form “A and/or B” means “(A), (B),or (A and B)”. For the purposes of this description, a phrase in theform “at least one of A, B, or C” means “(A), (B), (C), (A and B), (Aand C), (B and C), or (A, B and C)”.

As shown in the drawings and described throughout the followingdescription, as is traditional when referring to relative positioning ona surgical device, the term “proximal” refers to the end of theapparatus that is closer to the user or generator, while the term“distal” refers to the end of the apparatus that is farther away fromthe user or generator. The term “user” refers to any medicalprofessional (i.e., doctor, nurse, or the like) performing a medicalprocedure involving the use of aspects of the present disclosuredescribed herein.

As used in this description, the term “surgical device” generally refersto a surgical tool that imparts electrosurgical energy to treat tissue.Surgical devices may include, but are not limited to, needles, probes,catheters, endoscopic instruments, laparoscopic instruments, vesselsealing devices, surgical staplers, etc. The term “electrosurgicalenergy” generally refers to any form of electromagnetic, optical, oracoustic energy.

Electromagnetic (EM) energy is generally classified by increasingfrequency or decreasing wavelength into radio waves, microwaves,infrared, visible light, ultraviolet, X-rays and gamma-rays. As usedherein, the term “microwave” generally refers to electromagnetic wavesin the frequency range of 300 megahertz (MHz) (3×10⁸ cycles/second) to300 gigahertz (GHz) (3×10¹¹ cycles/second). As used herein, the term“RF” generally refers to electromagnetic waves having a lower frequencythan microwaves. As used herein, the term “ultrasound” generally refersto cyclic sound pressure with a frequency greater than the upper limitof human hearing.

As used in this description, the term “ablation procedure” generallyrefers to any ablation procedure, such as microwave ablation, radiofrequency (RF) ablation or microwave ablation-assisted resection. As itis used in this description, “energy applicator” generally refers to anydevice that can be used to transfer energy from a power generatingsource, such as a microwave or RF electrosurgical generator, to tissue.

As they are used in this description, the terms “power source” and“power supply” refer to any source (e.g., battery) of electrical powerin a form that is suitable for operating electronic circuits. As it isused in this description, “transmission line” generally refers to anytransmission medium that can be used for the propagation of signals fromone point to another. As used in this description, the terms “switch” or“switches” generally refers to any electrical actuators, mechanicalactuators, electro-mechanical actuators (rotatable actuators, pivotableactuators, toggle-like actuators, buttons, etc.), optical actuators, orany suitable device that generally fulfills the purpose of connectingand disconnecting electronic devices, or a component thereof,instruments, equipment, transmission line or connections andappurtenances thereto, or software.

As used in this description, “electronic device” generally refers to adevice or object that utilizes the properties of electrons or ionsmoving in a vacuum, gas, or semiconductor. As it is used herein,“electronic circuitry” generally refers to the path of electron or ionmovement, as well as the direction provided by the device or object tothe electrons or ions. As it is used herein, “electrical circuit” orsimply “circuit” generally refers to a combination of a number ofelectrical devices and conductors that when connected together, form aconducting path to fulfill a desired function. Any constituent part ofan electrical circuit other than the interconnections may be referred toas a “circuit element” that may include analog and/or digitalcomponents.

The term “generator” may refer to a device capable of providing energy.Such device may include a power source and an electrical circuit capableof modifying the energy outputted by the power source to output energyhaving a desired intensity, frequency, and/or waveform.

As it is used in this description, “user interface” generally refers toany visual, graphical, tactile, audible, sensory or other mechanism forproviding information to and/or receiving information from a user orother entity. The term “user interface” as used herein may refer to aninterface between a human user (or operator) and one or more devices toenable communication between the user and the device(s). Examples ofuser interfaces that may be employed in various embodiments of thepresent disclosure include, without limitation, switches,potentiometers, buttons, dials, sliders, a mouse, a pointing device, akeyboard, a keypad, joysticks, trackballs, display screens, varioustypes of graphical user interfaces (GUIs), touch screens, microphonesand other types of sensors or devices that may receive some form ofhuman-generated stimulus and generate a signal in response thereto. Asit is used herein, “computer” generally refers to anything thattransforms information in a purposeful way.

The systems described herein may also utilize one or more controllers toreceive various information and transform the received information togenerate an output. The controller may include any type of computingdevice, computational circuit, or any type of processor or processingcircuit capable of executing a series of instructions that are stored ina memory. The controller may include multiple processors and/ormulticore central processing units (CPUs) and may include any type ofprocessor, such as a microprocessor, digital signal processor,microcontroller, or the like. The controller may also include a memoryto store data and/or algorithms to perform a series of instructions.

Any of the herein described methods, programs, algorithms or codes maybe converted to, or expressed in, a programming language or computerprogram. A “Programming Language” and “Computer Program” is any languageused to specify instructions to a computer, and includes (but is notlimited to) these languages and their derivatives: Assembler, Basic,Batch files, BCPL, C, C+, C++, Delphi, Fortran, Java, JavaScript,Machine code, operating system command languages, Pascal, Perl, PL1,scripting languages, Visual Basic, metalanguages which themselvesspecify programs, and all first, second, third, fourth, and fifthgeneration computer languages. Also included are database and other dataschemas, and any other meta-languages. For the purposes of thisdefinition, no distinction is made between languages which areinterpreted, compiled, or use both compiled and interpreted approaches.For the purposes of this definition, no distinction is made betweencompiled and source versions of a program. Thus, reference to a program,where the programming language could exist in more than one state (suchas source, compiled, object, or linked) is a reference to any and allsuch states. The definition also encompasses the actual instructions andthe intent of those instructions.

Any of the herein described methods, programs, algorithms or codes maybe contained on one or more machine-readable media or memory. The term“memory” may include a mechanism that provides (e.g., stores and/ortransmits) information in a form readable by a machine such a processor,computer, or a digital processing device. For example, a memory mayinclude a read only memory (ROM), random access memory (RAM), magneticdisk storage media, optical storage media, flash memory devices, or anyother volatile or non-volatile memory storage device. Code orinstructions contained thereon can be represented by carrier wavesignals, infrared signals, digital signals, and by other like signals.

As it is used in this description, the phrase “treatment plan” refers toa selected ablation needle, energy level, and/or treatment duration toeffect treatment of a target. The term “target” refers to a region oftissue slated for treatment, and may include, without limitation,tumors, fibroids, and other tissue that is to be ablated. The phrase“ablation zone” refers to the area and/or volume of tissue that will beablated.

As it is used in this description, the phrase “computed tomography” (CT)or “computed axial tomography” (CAT) refer to a medical imaging methodemploying tomography created by computer processing. Digital geometryprocessing is used to generate a three-dimensional image of the insideof an object from a large series of two-dimensional X-ray images takenaround a single axis of rotation.

As it is used in this description, the term magnetic resonance imaging(MRI), nuclear magnetic resonance imaging (NMRI), or magnetic resonancetomography (MRT) refer to a medical imaging technique used in radiologyto visualize detailed internal structures. MRI makes use of the propertyof nuclear magnetic resonance (NMR) to image nuclei of atoms inside thebody. An MRI machine uses a powerful magnetic field to align themagnetization of some atomic nuclei in the body, while using radiofrequency fields to systematically alter the alignment of thismagnetization. This causes the nuclei to produce a rotating magneticfield detectable by the scanner and this information is recorded toconstruct an image of the scanned area of the body.

As it is used in this description, the term “three-dimensionalultrasound” or “3D ultrasound” refers to medical ultrasound techniqueproviding three dimensional images.

As it is used in this description, the phrase “digital imaging andcommunication in medicine” (DICOM) refers to a standard for handling,storing, printing, and transmitting information relating to medicalimaging. It includes a file format definition and a networkcommunications protocol. The communication protocol is an applicationprotocol that uses TCP/IP to communicate between systems. DICOM filescan be exchanged between two entities that are capable of receivingimage and patient data in DICOM format.

Any of the herein described systems and methods may transfer datatherebetween over a wired network, wireless network, point to pointcommunication protocol, a DICOM communication protocol, a transmissionline, a removable storage medium, and the like.

The systems described herein may utilize one or more sensors configuredto detect one or more properties of tissue and/or the ambientenvironment. Such properties include, but are not limited to: tissueimpedance, tissue type, tissue clarity, tissue compliance, temperatureof the tissue or jaw members, water content in tissue, jaw openingangle, water motality in tissue, energy delivery, and jaw closurepressure.

In an aspect of the present disclosure, a method of determining atreatment plan is provided. The method includes obtaining a plurality ofimages and rendering the plurality of images in three dimensions. Theplurality of images are automatically segmented to demarcate a targetarea, and a treatment plan is automatically determined based on thetarget area.

In the method, automatically segmenting the plurality of images includesselecting a seed point and creating a region of interest around the seedpoint. A first plurality of pixels in the region of interest is comparedto a predetermined threshold. A second plurality of pixels is selectedfrom the first plurality of pixels, wherein the second plurality ofpixels are connected to the seed point and are less than thepredetermined threshold. A geometric filter is applied to the secondplurality of pixels.

The method also includes determining if the second plurality of pixelsforms a predetermined object. If the second plurality of pixels does notform a predetermined object, the predetermined threshold is increasedand the steps of comparing a first plurality of pixels, selecting asecond plurality of pixels, applying a geometric filter, and determiningif the second plurality of pixels forms a predetermined object arerepeated.

Further, automatically determining a treatment plan includes performinga volumetric analysis on the target area, selecting a surgical device,and calculating an energy level and treatment duration based on thetarget area and the selected surgical device. The plurality of imagesare rendered and the target area and the treatment plan are displayed.

In addition, the method also includes automatically segmenting at leastone vessel or at least one organ. The treatment plan is adjusted basedon a proximity of the at least one vessel or organ to the target and theadjusted treatment plan is displayed.

In another aspect of the present disclosure, a method of navigating asurgical device using a fiducial pattern disposed on an ultrasounddevice and an image capture device disposed on the surgical device isprovided. In the method, an ultrasound image of a scan plane and afiducial image of the fiducial pattern are obtained. The fiducial imageis corrected for lens distortion and then a correspondence between thefiducial image and a model image is determined. A camera pose isestimated and a position of the surgical device is transformed to modelcoordinates. The ultrasound image and a virtual image of the surgicaldevice based on the model coordinates are then displayed.

In yet another aspect of the present disclosure, a method of tracking afirst device having an image capture device in relation to a seconddevice having a fiducial pattern is provided. The method includesobtaining a fiducial image of the fiducial pattern, correcting thefiducial image for lens distortion, finding correspondence between thefiducial image and a model image, estimating a camera pose, andtransforming a position of the surgical device to model coordinates.

The fiducial pattern includes a plurality of first unique identifiersdisposed in a region and a plurality of second unique identifiers. Themethod also includes finding the plurality of first unique identifiersby applying a first threshold to the fiducial image, performing aconnected component analysis, applying a geometric filter to determinethe weighted centroids of the plurality of first unique identifiers, andstoring the weighted centroids of the plurality of first uniqueidentifiers. In addition, the method includes finding the plurality ofsecond unique identifiers by inverting the fiducial image, applying asecond threshold to the inverted fiducial image, performing a connectedcomponent analysis, applying a geometric filter to determine theweighted centroids of the plurality of second unique identifiers and todetermine the region having the plurality of first unique identifiers,and storing the weighted centroids of the plurality of secondidentifiers and the region having the plurality of first uniqueidentifiers. The first threshold and second threshold are dynamicthresholds.

Correspondence between the fiducial image and the model image is foundby selecting a plurality of first unique identifiers from the fiducialimage, arranging the plurality of first unique identifiers in clockwiseorder, arranging a plurality of model fiducials in clockwise order, andcomputing a planar homography. The plurality of model fiducials aretransformed into image coordinates using the computed planar homography.A model fiducial from the plurality of model fiducials is found thatmatches the fiducial image, and the residual error is computed.

Selecting a plurality of first unique identifiers from the fiducialimage is done by selecting the plurality of first unique identifiers,selecting the region having the plurality of first unique identifiers,counting the number of first unique identifiers in the selected region,and comparing the number of first unique identifiers in the selectedregion to a predetermined number. If the number of first uniqueidentifiers in the selected region equals the predetermined number, themethod proceeds to arranging the plurality of first unique identifiersin clockwise order. If the number of first unique identifiers in theselected region does not equal the predetermined number, a new region isselected and the number of first unique identifiers in the new region iscounted.

In yet another aspect of the present disclosure, a planning andnavigation method is provided. The planning and navigation methodincludes obtaining a plurality of images, rendering the plurality ofimages in three dimensions, automatically segmenting the plurality ofimages to demarcate a target area, and automatically determining atreatment plan based on the target area. The planning and navigationmethod also includes obtaining an ultrasound image of a scan planeincluding the target, obtaining a fiducial image of a fiducial patterndisposed on an ultrasound device using an image capture device on asurgical device, correcting the fiducial image for lens distortion,finding correspondence between the fiducial image and a model image,estimating a camera pose, transforming a position of the surgical deviceto model coordinates, displaying the ultrasound image and a virtualimage of the surgical device based on the model coordinates, navigatingthe surgical device to the target using the displayed ultrasound imageand the virtual image, and treating the target based on the treatmentplan.

In the method, automatically segmenting the plurality of images includesselecting a seed point and creating a region of interest around the seedpoint. A first plurality of pixels in the region of interest is comparedto a predetermined threshold. A second plurality of pixels is selectedfrom the first plurality of pixels, wherein the second plurality ofpixels are connected to the seed point and are less than thepredetermined threshold. A geometric filter is applied to the secondplurality of pixels.

The planning and navigation method also includes determining if thesecond plurality of pixels forms a predetermined object. If the secondplurality of pixels does not form a predetermined object, thepredetermined threshold is increased and the steps of comparing a firstplurality of pixels, selecting a second plurality of pixels, applying ageometric filter, and determining if the second plurality of pixelsforms a predetermined object are repeated.

In another aspect, automatically determining a treatment plan includesperforming a volumetric analysis on the target area, selecting asurgical device, and calculating an energy level and treatment durationbased on the target area and the selected surgical device. The pluralityof images are rendered and the target area and the treatment plan aredisplayed.

In addition, the method also includes automatically segmenting at leastone vessel or at least one organ. The treatment plan is adjusted basedon a proximity of the at least one vessel or organ to the target and theadjusted treatment plan is displayed.

The fiducial pattern includes a plurality of first unique identifiersdisposed in a region and a plurality of second unique identifiers. Themethod also includes finding the plurality of first unique identifiersby applying a first threshold to the fiducial image, performing aconnected component analysis, applying a geometric filter to determinethe weighted centroids of the plurality of first unique identifiers, andstoring the weighted centroids of the plurality of first uniqueidentifiers. In addition, the method includes finding the plurality ofsecond unique identifiers by inverting the fiducial image, applying asecond threshold to the inverted fiducial image, performing a connectedcomponent analysis, applying a geometric filter to determine theweighted centroids of the plurality of second unique identifiers and todetermine the region having the plurality of first unique identifiers,and storing the weighted centroids of the plurality of secondidentifiers and the region having the plurality of first uniqueidentifiers. The first threshold and second threshold are dynamicthresholds.

Correspondence between the fiducial image and the model image is foundby selecting a plurality of first unique identifiers from the fiducialimage, arranging the plurality of first unique identifiers in clockwiseorder, arranging a plurality of model fiducials in clockwise order, andcomputing a planar homography. The plurality of model fiducials aretransformed into image coordinates using the computed planar homography.A model fiducial from the plurality of model fiducials is found thatmatches the fiducial image and the residual error is computed.

Selecting a plurality of first unique identifiers from the fiducialimage is done by selecting the plurality of first unique identifiers,selecting the region having the plurality of first unique identifiers,counting the number of first unique identifiers in the selected region,and comparing the number of first unique identifiers in the selectedregion to a predetermined number. If the number of first uniqueidentifiers in the selected region equals the predetermined number, themethod proceeds to arranging the plurality of first unique identifiersin clockwise order. If the number of first unique identifiers in theselected region does not equal the predetermined number, a new region isselected and the number of first unique identifiers in the new region iscounted.

In yet another aspect of the present disclosure, a planning system isprovided. The planning system includes a memory configured to store aplurality of images. The planning system also includes a controllerconfigured to render the plurality of images in three dimensions,automatically segment the plurality of images to demarcate a targetarea, and automatically determine a treatment plan based on the targetarea. A display is provided to display the rendered plurality of imagesand the target area.

In the planning system, the controller performs a volumetric analysis todetermine a treatment plan. The planning system may also include aninput means configured to adjust the treatment plan. The displayprovides a graphical user interface.

The controller may segment at least one vessel and adjust the treatmentplan based on the proximity of the vessel to the target. The controllermay segment at least one organ and adjust the treatment plan based on aposition of the target in relation to the organ.

In yet another aspect of the present disclosure, a navigation system isprovided. The navigation system includes an ultrasound device having afiducial pattern disposed thereon configured to obtain an ultrasoundimage in a scan plane and a surgical instrument having an image capturedevice configured to capture a fiducial image of the fiducial pattern. Acontroller is configured to receive the ultrasound image and thefiducial image, wherein the controller determines a position of thesurgical instrument in relation to the scan plane based on the fiducialimage and a display is configured to display the ultrasound image and avirtual image of the surgical instrument based on the position of thesurgical instrument in relation to the scan plane.

In the navigation system, the fiducial pattern is affixed to a knownlocation on the ultrasound device and the image capture device isaffixed to a known location on the surgical instrument. The fiducialpattern has a plurality of markings of known characteristics andrespective relative positions that reside within a known topology. Thecontroller corresponds the fiducial image to a model image, estimates acamera pose, and transforms the surgical instrument to modelcoordinates. The controller also corrects the fiducial image for lensdistortion. Additionally, the controller can recognize a topology withinthe fiducial marker where the topology references two or moreindependent unique identifiers located in known positions on a singlepattern on a marker.

In yet another aspect of the present disclosure, a fiducial trackingsystem is provided. The fiducial tracking system includes a first devicehaving a fiducial pattern disposed thereon and a second device having animage capture device disposed thereon. The image capturing device isconfigured to obtain a fiducial image of the fiducial pattern. Acontroller is also provided that receives the fiducial image, correctsthe fiducial image for lens distortion, finds correspondence between thefiducial image and a model image, estimates a camera pose, andtransforms a position of the surgical device to model coordinates.

In the fiducial tracking system, the fiducial pattern includes aplurality of first unique identifiers disposed in a region and aplurality of second unique identifiers.

The controller finds the plurality of first unique identifiers byapplying a first threshold to the fiducial image, performing a connectedcomponent analysis, applying a geometric filter to determine theweighted centroids of the plurality of first unique identifiers, andstoring the weighted centroids of the plurality of first uniqueidentifiers.

The controller finds the plurality of second unique identifiers byinverting the fiducial image, applying a second threshold to theinverted fiducial image, performing a connected component analysis,applying a geometric filter to determine the weighted centroids of theplurality of second unique identifiers and to determine the regionhaving the plurality of first unique identifiers, and storing theweighted centroids of the plurality of second identifiers and the regionhaving the plurality of first unique identifiers.

The controller may find correspondence between the fiducial image andthe model image by selecting a plurality of first unique identifiersfrom the fiducial image, arranging the plurality of first uniqueidentifiers in clockwise order, arranging a plurality of model fiducialsin clockwise order, computing a planar homography, transforming theplurality of model fiducials into image coordinates using the computedplanar homography, finding a model fiducial from the plurality of modelfiducials that matches the fiducial image, and computing the residualerror.

The controller may also select a plurality of first unique identifiersfrom the fiducial image by selecting the plurality of first uniqueidentifiers, selecting the region having the plurality of first uniqueidentifiers, counting the number of first unique identifiers in theselected region, and comparing the number of first unique identifiers inthe selected region to a predetermined number. If the number of firstunique identifiers in the selected region equals the predeterminednumber, the method proceeds to arranging the plurality of first uniqueidentifiers in clockwise order. If the number of first uniqueidentifiers in the selected region does not equal the predeterminednumber, a new region is selected and the number of first uniqueidentifiers in the new region is counted. The first threshold or thesecond threshold is a dynamic threshold.

In yet another aspect of the present disclosure, a planning andnavigation system is provided. The planning system includes a memoryconfigured to store a plurality of images and a first controllerconfigured to render the plurality of images in three dimensions,automatically segment the plurality of images to demarcate a targetarea, and automatically determine a treatment plan based on the targetarea. The navigation system includes an ultrasound device having afiducial pattern disposed thereon and configured to obtain an ultrasoundimage in a scan plane, a surgical device having a image capture deviceconfigured to capture a fiducial image of the fiducial pattern, and asecond controller configured to receive the ultrasound image and thefiducial image, wherein the controller determines a position of thesurgical device in relation to the scan plane based on the fiducialimage. The planning and navigation system also includes a thirdcontroller configured to receive the rendered plurality of images, thetarget area, the treatment plan, the ultrasound image, and the positionof the surgical device in relation to the scan plane, and a displayconfigured to display a first display having the rendered plurality ofimages, the target area, and the treatment plan, and configured todisplay a second display having the ultrasound image, a virtual image ofthe surgical device based on the position of the surgical device inrelation to the scan plane, and the treatment plan.

In the planning and navigation system, the first display and the seconddisplay are displayed on a single screen and may be displayedsimultaneously or a user can switch between the first display and thesecond display. Alternatively, the display may have two screens and thefirst display is displayed on a first screen and the second display isdisplayed on a second screen. The display may provide a graphical userinterface.

The first controller performs a volumetric analysis to determine atreatment plan and an input means may be provided to adjust thetreatment plan. The first controller may segment at least one vessel andadjust the treatment plan based on the proximity of the vessel to thetarget. The first controller may segment at least one organ and adjustthe treatment plan based on a position of the target in relation to theorgan.

In the planning and navigation system, the fiducial pattern is affixedto a known location on the ultrasound device and the image capturedevice is affixed to a known location on the surgical device. Thefiducial pattern has a plurality of markings of known characteristicsand relative positions that reside within a known topology.

The second controller corresponds the fiducial image to a model image,estimates a camera pose, and transforms the surgical device to modelcoordinates. The second controller also corrects the fiducial image forlens distortion. Additionally, the second controller can recognize atopology within the fiducial marker where the topology references two ormore independent unique identifiers located in known positions on asingle pattern on a marker.

In yet another aspect of the present disclosure, a planning andnavigation system is provided. The planning system includes a memoryconfigured to store a plurality of images and a first controllerconfigured to render the plurality of images in three dimensions,automatically segment the plurality of images to demarcate a targetarea, and automatically determine a treatment plan based on the targetarea. The navigation system includes an ultrasound device having afiducial pattern disposed thereon and configured to obtain an ultrasoundimage in a scan plane, an ablation needle having a image capture deviceconfigured to capture a fiducial image of the fiducial pattern, and asecond controller configured to receive the ultrasound image and thefiducial image, wherein the controller determines a position of theablation needle in relation to the scan plane based on the fiducialimage. The planning and navigation system also includes a thirdcontroller configured to receive the rendered plurality of images, thetarget area, the treatment plan, the ultrasound image, and the positionof the ablation needle in relation to the scan plane and a displayconfigured to display a first display having the rendered plurality ofimages, the target area, and the treatment plan, and configured todisplay a second display having the ultrasound image, a virtual image ofthe ablation needle based on the position of the ablation needle inrelation to the scan plane, and the treatment plan.

In the planning and navigation system, the first display and the seconddisplay are displayed on a single screen and may be displayedsimultaneously, or, a user can switch between the first display and thesecond display. Alternatively, the display may have two screens and thefirst display is displayed on a first screen and the second display isdisplayed on a second screen. The display may provide a graphical userinterface.

The first controller performs a volumetric analysis to determine atreatment plan and an input means may be provided to adjust thetreatment plan. The first controller may segment at least one vessel andadjust the treatment plan based on the proximity of the vessel to thetarget. The first controller may segment at least one organ and adjustthe treatment plan based on a position of the target in relation to theorgan.

In the planning and navigation system, the fiducial pattern is affixedto a known location on the ultrasound device and the image capturedevice is affixed to a known location on the ablation needle. Thefiducial pattern has a plurality of markings of known characteristicsand relative positions that reside within a known topology.

The second controller corresponds the fiducial image to a model image,estimates a camera pose, and transforms the ablation needle to modelcoordinates. The second controller also corrects the fiducial image forlens distortion. Additionally, the second controller can recognize atopology within the fiducial marker where the topology references two ormore independent unique identifiers located in known positions on asingle pattern on a marker.

In yet another aspect of the present disclosure, an ablation planningand navigation system is provided. The planning system includes a memoryconfigured to store a plurality of images and a first controllerconfigured to render the plurality of images in three dimensions,automatically segment the plurality of images to demarcate a targetarea, and automatically determine a treatment plan based on the targetarea. The navigation system includes an ultrasound device having afiducial pattern disposed thereon and configured to obtain an ultrasoundimage in a scan plane, an ablation needle having a image capture deviceconfigured to capture a fiducial image of the fiducial pattern, and asecond controller configured to receive the ultrasound image and thefiducial image, wherein the controller determines a position of theablation needle in relation to the scan plane based on the fiducialimage. The ablation planning and navigation system also includes a thirdcontroller configured to receive the rendered plurality of images, thetarget area, the treatment plan, the ultrasound image, and the positionof the ablation needle in relation to the scan plane and a displayconfigured to display a first display having the rendered plurality ofimages, the target area, and the treatment plan and configured todisplay a second display having the ultrasound image, a virtual image ofthe ablation needle based on the position of the ablation needle inrelation to the scan plane, and the treatment plan.

In the ablation planning and navigation system, the first display andthe second display are displayed on a single screen and may be displayedsimultaneously or a user can switch between the first display and thesecond display. Alternatively, the display may have two screens and thefirst display is displayed on a first screen and the second display isdisplayed on a second screen. The display may provide a graphical userinterface.

The first controller performs a volumetric analysis to determine atreatment plan and an input means may be provided to adjust thetreatment plan. The first controller may also segment at least onevessel and adjust the treatment plan based on the proximity of thevessel to the target, or, the first controller may segment at least oneorgan and adjust the treatment plan based on a position of the target inrelation to the organ.

In the ablation planning and navigation system, the fiducial pattern isaffixed to a known location on the ultrasound device and the imagecapture device is affixed to a known location on the ablation needle.The fiducial pattern has a plurality of markings of knowncharacteristics and relative positions that reside within a knowntopology.

The second controller corresponds the fiducial image to a model image,estimates a camera pose, and transforms the ablation needle to modelcoordinates. The second controller also corrects the fiducial image forlens distortion. Additionally, the second controller can recognize atopology within the fiducial marker where the topology references two ormore independent unique identifiers located in known positions on asingle pattern on a marker.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features, and advantages of the presentdisclosure will become more apparent in light of the following detaileddescription when taken in conjunction with the accompanying drawings inwhich:

FIG. 1 is a system block diagram of a planning and navigation systemaccording to an embodiment of the present disclosure;

FIGS. 2A and 2B are schematic diagrams of an ablation needle accordingto an embodiment of the present disclosure;

FIG. 3 is a schematic diagram of a radiation pattern of the ablationneedle of FIGS. 2A and 2B;

FIG. 4 is a schematic diagram of a planning system according to anembodiment of the present disclosure;

FIG. 5 is a flowchart depicting overall operation of the planning systemaccording to an embodiment of the present disclosure;

FIGS. 6 and 7 are schematic diagrams of graphical user interfaces usedin the planning system in accordance with an embodiment of the presentdisclosure;

FIG. 8 is a flowchart depicting an algorithm for image segmentation andinverse planning according to an embodiment of the present disclosure;

FIG. 9 is a flowchart depicting an algorithm for segmenting a noduleaccording to an embodiment of the present disclosure;

FIGS. 10A-10B are graphical representations of relationships betweenablation zones and energy delivery;

FIG. 11A is a schematic diagram of a relationship between a vessel and atarget according to another embodiment of the present disclosure;

FIG. 11B is a graphical representation of an alternate dosing curveaccording to another embodiment of the present disclosure;

FIGS. 12A-12C are schematic diagrams of a planning method according toanother embodiment of the present disclosure;

FIG. 13 is a schematic diagram of a navigation system according to anembodiment of the present disclosure;

FIGS. 14A and 14B are schematic diagrams of graphical user interfacesused in the navigation system of FIG. 13;

FIG. 15 is a flowchart depicting a fiducial tracking algorithm accordingto an embodiment of the present disclosure;

FIGS. 16A and 16B depict an image taken by a camera and a correctedversion of the image, respectively;

FIG. 17 is a flowchart depicting an algorithm for finding white circlesaccording to an embodiment of the present disclosure;

FIGS. 18A-18C depict intermediate image results of the algorithmdepicted in FIG. 17;

FIG. 19 is a flowchart depicting an algorithm for finding black circlesand black regions according to an embodiment of the present disclosure;

FIGS. 20A-20D depict intermediate image results of the algorithmdepicted in FIG. 19;

FIG. 21A is a flowchart depicting a correspondence algorithm accordingto an embodiment of the present disclosure;

FIG. 21B is a flowchart depicting an algorithm for applying a topologyconstraint according to an embodiment of the present disclosure;

FIG. 22A-22D are a schematic diagrams of fiducial models used in thealgorithm of FIG. 21A;

FIG. 23 is a schematic diagram of an integrated planning and navigationsystem according to another embodiment of the present disclosure;

FIG. 24 is a schematic diagram of an integrated planning and navigationsystem according to yet another embodiment of the present disclosure;

FIGS. 25A and 25B are schematic diagrams of a navigation system suitablefor use with the system of FIG. 24; and

FIGS. 26-29 are schematic diagrams of graphical user interfaces used inthe system of FIG. 24 in accordance with various embodiments of thepresent disclosure.

DETAILED DESCRIPTION

Particular embodiments of the present disclosure are describedhereinbelow with reference to the accompanying drawings; however, it isto be understood that the disclosed embodiments are merely examples ofthe disclosure and may be embodied in various forms. Well-knownfunctions or constructions are not described in detail to avoidobscuring the present disclosure in unnecessary detail. Therefore,specific structural and functional details disclosed herein are not tobe interpreted as limiting, but merely as a basis for the claims and asa representative basis for teaching one skilled in the art to variouslyemploy the present disclosure in virtually any appropriately detailedstructure. Like reference numerals may refer to similar or identicalelements throughout the description of the figures.

Turning to the figures, FIG. 1 depicts an overview of a planning andnavigation system according to various embodiments of the presentdisclosure. As shown in FIG. 1, pre-operative images 15 of a patient “P”are captured via an image capture device 10. Image capture device 10 mayinclude, but is not limited to, a MRI device, a CAT device, or anultrasound device that obtains two-dimensional (2D) or three-dimensional(3D) images. Image capture device 10 stores pre-operative images 15 thatare transferred to planning system 100. Pre-operative images 15 may betransferred to planning system 100 by uploading images 15 to a network,transmitting images 15 to planning system 100 via a wirelesscommunication means, and/or storing images 15 on a removable memory thatis inserted into planning system 100. In an embodiment of the presentdisclosure, pre-operative images 15 are stored in a DICOM format. Insome embodiments, image capture device 10 and planning system 100 may beincorporated into a standalone unit.

Planning system 100, which is described in more detail below, receivesthe pre-operative images 15 and determines the size of a target. Basedon the target size and a selected surgical device, planning system 100determines settings that include an energy level and a treatmentduration to effect treatment of the target.

Navigation system 200, which is described in more detail below, utilizesa fiducial pattern disposed on a medical imaging device (e.g., anultrasound imaging device) to determine an intracorporeal position of ansurgical device. The intracorporeal position of the surgical device isdisplayed on a display device in relation to an image obtained by themedical imaging device. Once the surgical device is positioned in thevicinity of the target, the user effects treatment of the target basedon the treatment zone settings determined by the planning system.

In some embodiments, a user determines the treatment zone settings usingplanning system 100 and utilizes the treatment zone settings ineffecting treatment using navigation system 200. In other embodiments,the planning system 100 transmits the treatment zone settings tonavigation system 200 to automatically effect treatment of the targetwhen the surgical device is in the vicinity of the target. Additionally,in some embodiments, planning system 100 and navigation system 200 arecombined into a single standalone system. For instance, a singleprocessor and a single user interface may be used for planning system100 and navigation system 200, a single processor and multiple userinterfaces may be used to for planning system 100 and navigation system200, or multiple processors and a single user interface may be used forplanning system 100 and navigation system 200.

FIG. 2A shows an example of a surgical device in accordance with anembodiment of the present disclosure. Specifically, FIG. 2A shows a sideview of a variation on an ablation needle 60 with an electrical choke 72and FIG. 2B shows a cross-section side view 2B-2B from FIG. 2A. Ablationneedle 60 shows radiating portion 62 electrically attached via feedline(or shaft) 64 to a proximally located coupler 66. Radiating portion 62is shown with sealant layer 68 coated over section 62. Electrical choke72 is shown partially disposed over a distal section of feedline 64 toform electrical choke portion 70, which is located proximally ofradiating portion 62.

To improve the energy focus of the ablation needle 60, the electricalchoke 72 is used to contain field propagation or radiation pattern tothe distal end of the ablation needle 60. Generally, the choke 72 isdisposed on the ablation needle 60 proximally of the radiating section.The choke 72 is placed over a dielectric material that is disposed overthe ablation needle 60. The choke 72 is a conductive layer that may becovered by a tubing or coating to force the conductive layer to conformto the underlying ablation needle 60, thereby forming an electricalconnection (or short) more distally and closer to the radiating portion62. The electrical connection between the choke 72 and the underlyingablation needle 60 may also be achieved by other connection methods suchas soldering, welding, brazing, crimping, use of conductive adhesives,etc. Ablation needle 60 is electrically coupled to a generator thatprovides ablation needle 60 with electrosurgical energy.

FIG. 3 is a cross-sectional view of an embodiment of the ablation needle60 shown with a diagrammatic representation of an emitted radiationpattern in accordance with the present disclosure.

FIGS. 4 to 12C describe the operation of planning system 100 inaccordance with various embodiments of the present disclosure. Turningto FIG. 4, planning system 100 includes a receiver 102, memory 104,controller 106, input device 108 (e.g., mouse, keyboard, touchpad,touchscreen, etc.), and a display 110. During operation of the planningsystem 100, receiver 102 receives pre-operative images 15 in DICOMformat and stores the images in memory 104. Controller 106 thenprocesses images 15, which is described in more detail below, anddisplays the processed images on display 110. Using input device 108, auser can navigate through the images 15, select one of the images fromimages 15, select a seed point on the selected image, select an ablationneedle, adjust the energy level, and adjust the treatment duration. Theinputs provided by input device 108 are displayed on display 110.

FIG. 5 depicts a general overview of an algorithm used by planningsystem 100 to determine a treatment plan. As shown in FIG. 5, in step120, images in a DICOM format are acquired via a wireless connection, anetwork, or by downloading the images from a removable storage mediumand stored in memory 104. Controller 106 then performs an automaticthree dimensional (3D) rendering of the images 15 and displays a 3Drendered image (as shown in FIG. 6) in step 122. In step 124, imagesegmentation is performed to demarcate specific areas of interest andcalculate volumetrics of the areas of interest. As described below,segmentation can be user driven or automatic. In step 126, thecontroller performs an inverse planning operation, which will also bedescribed in more detail below, to determine a treatment algorithm totreat the areas of interest. The treatment algorithm may includeselection of a surgical device, energy level, and/or duration oftreatment. Alternatively, a user can select the surgical device, energylevel, and/or duration of treatment to meet the intentions of a treatingphysician that would include a “margin value” in order to treat thetarget and a margin of the surrounding tissue.

FIGS. 6 and 7 depict graphical user interfaces (GUIs) that may bedisplayed on display 110. As shown in FIGS. 6 and 7, each GUI is dividedinto a number of regions (e.g., regions 132, 134, and 136) fordisplaying the rendered DICOM images. For example, region 132 shows animage of patient “P” along a transverse cross-section and region 134shows an image of patient “P” along a coronal cross-section. Region 136depicts a 3D rendering of patient “P”. In other embodiments, a sagittalcross-section may also be displayed on the GUI. The GUI allows a user toselect different ablation needles in drop down menu 131. The GUI alsoallows a user to adjust the power and time settings in regions 133 and135, respectively. Additionally, the GUI has a number of additionaltools in region 137 that include, but are not limited to, a planningtool that initiates the selection of a seed point, a contrast tool, azoom tool, a drag tool, a scroll tool for scrolling through DICOMimages, and a 3D Render tool for displaying the volume rendering of theDICOM dataset.

The flowchart of FIG. 8 depicts the basic algorithm for performing theimage segmentation step 124 and the inverse planning step 126. As shownin FIG. 8, a user selects a seed point in step 140 (see FIG. 6 where across hair is centered on the target “T” in regions 132 and 134). Afterthe seed point is manually selected, planning system 100 segments anodule to demarcate a volume of interest in step 142. In otherembodiments, the seed point may be automatically detected based on theintensity values of the pixels.

FIG. 9 depicts a flowchart of an algorithm used to segment a nodule. Asshown in FIG. 9, once a seed point is identified in step 151, thealgorithm creates a Region of Interest (ROI) in step 152. For example,the ROI may encompass a volume of 4 cm³. In step 153, a connectedthreshold filter applies a threshold and finds all the pixels connectedto the seed point in the DICOM images stored in memory 104. For example,the threshold values may start at −400 Houndsfields Units (HU) and endat 100 HU when segmenting lung nodules.

In step 154, controller 106 applies a geometric filter to compute thesize and shape of an object. The geometric filter enables themeasurement of geometric features of all objects in a labeled volume.This labeled volume can represent, for instance, a medical imagesegmented into different anatomical structures. The measurement ofvarious geometric features of these objects can provide additionalinsight into the image.

The algorithm determines if a predetermined shape is detected in step155. If a predetermined shape is not detected, the algorithm proceeds tostep 156 where the threshold is increased by a predetermined value. Thealgorithm repeats steps 153 to 155 until a predetermined object isdetected.

Once a predetermined object is detected, the algorithm ends in step 157and the planning system 100 proceeds to step 144 to perform volumetricanalysis. During the volumetric analysis, the following properties ofthe spherical object may be calculated by controller 106: minimumdiameter; maximum diameter; average diameter; volume; sphericity;minimum density; maximum density; and average density. The calculatedproperties may be displayed on display 110 as shown in region 139 ofFIG. 7. The volumetric analysis may use a geometric filter to determinea minimum diameter, a maximum diameter, volume, elongation, surfacearea, and/or sphericity. An image intensity statistics filter may alsobe used in conjunction with the geometric filter in step 144. The imageintensity statistics filter calculates a minimum density, maximumdensity, and average density.

In step 146, power and time settings are calculated for a demarcatedtarget. FIG. 10 depicts various graphs of the relation ship betweenenergy deposited into tissue and the resulting ablation zone for a giventime period. This relationship allows for inverse planning byconsidering the dimension and characteristics of a target tissue (i.e.,tumors, fibroids, etc.) and the energy dose/antenna design of a specificablation needle. Table 1 below shows an example of a relationshipbetween ablation volume, power, and time for an ablation needle.

TABLE 1 Ablation Volume (cm³) Power (W) Time (s) 6 140 1 22 140 3 41 1405 31 110 5 23 80 5

Using the values in Table 1, a linear equation can be derived from thetable to compute optimal power and time settings. For example, using alinear regression analysis, Table 1 provides the following equation:

Volume=0.292381*Power+8.685714*Time−44.0762  (1)

which can be written as

Power=(Volume−8.685714*Time+44.0762)/0.292381.  (2)

The desired volume can be calculated using the maximum diameter from thevolumetric analysis plus a 1 centimeter margin as follows:

DesiredVolume=4/3*pi*DesiredRadiuŝ3  (3)

where the desired radius is calculated as follows:

DesiredRadius=MaximumNoduleDiameter/2+Margin.  (4)

Substituting the desired volume into equation (1) or (2) leaves twounknowns, power and time. Using equation (2) controller 106 can solvefor power by substituting values for time. Controller 106 chooses thesmallest value for time that maintains power below 70 W, or some otherpredetermined value, so that the user can perform the procedure asquickly as possible while keeping power in a safe range.

Once the power and time are calculated 146, the power and time aredisplayed on display 110 as shown in FIG. 7 (see 133 and 135). A usercan adjust the calculated power and/or time using controls 133 and 135,respectively, to adjust the treatment zone 138 a and/or margin 138 b.

Memory 104 and/or controller 106 may store a number of equations thatcorrespond to different surgical devices. When a user selects adifferent surgical devices in drop down menu 131, controller 106 canperform the same analysis described above to determine the smallestvalue for time that keeps the power below 70 W or some otherpredetermined value.

Although the above described procedure describes the use of a singleseed point to determine a predetermined object, some targets may have anirregular shape that can not be treated by the predetermined treatmentzone without causing damage to other tissue. In such instances, multipleseed points may be used to create an irregular shaped treatment planusing a single surgical device that is repositioned in a number ofplaces or multiple surgical devices that may be used concurrently totreat an irregularly shaped region.

In other embodiments, memory 104 and/or controller 106 may store acatalog of surgical devices and treatment zone performance, whichincludes power, time, number of instruments, and spacing of instrumentsrequired to achieve treatment zones ex vivo or in vivo. Based on theresults of the image segmentation and volumetric analysis, thecontroller may automatically select device types, numbers of devices,spacing of multiple devices, and/or power and time settings for eachdevice to treat the ROI. Alternatively, a user can manually selectdevice types, numbers of devices, spacing of multiple devices, powerand/or time settings for each device to treat the ROI using the GUI togenerate a treatment plan.

In another embodiment according to the present disclosure, planningsystem 100 may also segment organs and other vital structures inaddition to targets. Segmentation of organs and other structures, suchas vessels, are used to provide a more advanced treatment plan. Asdescribed above with regard to FIG. 10, treatment zones correlate toenergy delivery in a regular fashion. Further, it is known that vesselsgreater than three (3) millimeters may negatively affect treatment zoneformation. Segmentation of a vessel would allow the interaction betweenthe vessels and the target to be estimated, including the vesseldiameter (D1) and distance (D2) (see FIG. 11A) between the vessel and aproposed target. This interaction may be estimated manually by a user orautomatically by controller 106. Using the vessel diameter D1 and thedistance D2, planning system 100 may automatically suggest an alternatedose curve to be used for treatment purposes as shown in FIG. 11B.Alternatively, controller 106 may provide a recommendation to the uservia display 110 to move the treatment zone. Additionally, a differenttreatment zone projection could be displayed on display 110. Further, inthe compute power and time settings step 146 of FIG. 8, the controllercould leverage different curves depending on the vessel's diameter anddistance to the target area.

FIGS. 12A-12C depict an advanced treatment planning using organsegmentation. Segmentation of an organ allows for at least twoadvantages in planning a course of treatment. In a first instance,minimally invasive treatments are often chosen to be organ sparing. Bysegmenting the organ, controller 106 can calculate the organ volume 160and subtract the determined ablation zone 162 to determine the volume oforgan being spared 164 as shown in FIG. 12A. If controller 106determines that volume of organ being spared is too low, controller 106may alert a user that an alternate treatment plan is needed or it maysuggest an alternate treatment plan.

FIGS. 12B and 12C depict a treatment plan for a target “T” located onthe surface of an organ. Conventionally, treatment near an organ surfaceis often avoided or additional techniques may be required to separatethe organ from other organs before treatment can be performed. Inanother embodiment in accordance with the present disclosure, after theorgan is segmented, the position of a target “T” can also be determined.If the treatment zone 162 in the treatment plan projects outside thesurface of the organ and the target “T” is located on the surface,controller 106 may alert the user that treatment zone 162 may affectother organs and/or structures in the vicinity of the target “T” andthat the treatment plan needs to be altered. In another embodiment,controller 106 may automatically make recommendations to the userindicating the surgical device, energy level, duration of treatment.Controller 106 may also suggest a smaller treatment zone 162 as shown inFIG. 12B or it may suggest moving the treatment zone 162 as shown inFIG. 12C.

In other embodiments, after targets, tissues, organs, and otherstructures are segmented, known tissue properties can be attributed tothese structures. Such tissue properties include, but are not limitedto, electrical conductivity and permittivity across frequency, thermalconductivity, thermal convection coefficients, and so forth. Theplanning algorithm of FIG. 8 may use the tissue properties attributed tothe segmented tumors, tissues, organs, and other structures to solve thePennes bioheat equation in order to calculate a dose required to ablatea selected target. Keys to successful implementation of this morecomprehensive solution using the bioheat equation include: utilizingknown tissue properties at steady-state to predict an initial spatialtemperature profile, utilizing tissue properties as temperature rises toadjust spatial properties in accordance with temperature elevation, andutilizing tissue properties at liquid-gas phase transition.

Turning to FIG. 13, a navigation system in accordance with an embodimentof the present disclosure is shown generally as 200. Generally,navigation system 200 incorporates a reference patch or fiducial patch204 that is affixed to an ultrasound transducer 202. Fiducial patch 204may be printed on ultrasound transducer 202, attached to ultrasoundtransducer 202 via an adhesive, or removably coupled to ultrasoundtransducer 202. In some embodiments, the fiducial patch is disposed on asupport structure that is configured to be removably affixed, e.g.,“clipped onto”, the housing of an ultrasound transducer. Ultrasoundtransducer 202 is coupled to an ultrasound generator 210 that generatesacoustic waves. Ultrasound transducer 202 and ultrasound generator 210may be incorporated into a standalone unit. Ultrasound transducer 202emits the acoustic waves toward patient “P”. The acoustic waves reflectoff various structures in patient “P” and are received by ultrasoundtransducer 202. Ultrasound transducer 202 transmits the reflectedacoustic waves to an ultrasound generator 210 that converts thereflected acoustic waves into a two dimensional (2D) image in real time.The 2D image is transmitted to a controller 212. Controller 212processes the 2D image and displays the 2D image as image 218 includingtarget 220 on display 214. Image 218 is a real time representation ofscan plane “S” which may include target “T”.

The navigation system also incorporates a camera 208 affixed to ansurgical device 206. The camera 208 captures an image of fiducial patch204 in real time in order to determine the position of the surgicaldevice 206 in relation to the scan plane “S”. In particular, fiducialpatch 204 has a defined spatial relationship to scan plane “S”. Thisdefined spatial relationship is stored in controller 212. Camera 208also has a known spatial relationship to surgical device 206 that isstored in controller 212. In order to determine the spatial relationshipbetween surgical device 206 and scan plane “S”, camera 208 captures animage of fiducial patch 204 and transmits the image to controller 212.Using the image of the fiducial patch 204, controller 212 can calculatethe spatial relationship between the surgical device 206 and the scanplane “S”.

After controller 212 determines the spatial relationship between thesurgical device 206 and scan plane “S”, controller 212 displays thatrelationship on display 214. As shown in FIG. 13, display 214 includesan image 218 of scan plane “S” including a target image 220 of target“T”. Additionally, controller 212 superimposes a virtual image 206 a ofsurgical device 206 in relation to image 218 to indicate the position ofthe surgical device 206 in relation to scan plane “S”. Based on theangle and position of the ablation needle 206, controller 212 cancalculate a trajectory of the surgical device 206 and display thecalculated trajectory shown generally as 216. In some embodiments, acrosshair or target may be superimposed on image 218 to indicate wherethe surgical device 206 will intersect the scan plane “S”. In otherembodiments, the calculated trajectory 216 may be shown in red or greento indicate the navigation status. For instance, if surgical device 206is on a path that will intersect target “T”, calculated trajectory 216will be shown in green. If surgical device 206 is not on a path thatwill intersect target “T”, calculated trajectory 216 will be shown inred.

Controller 212 can also be controlled by a user to input the surgicaldevice type, energy level, and treatment duration. The surgical devicetype, energy level, and treatment duration can be displayed on display214 as shown in FIG. 14A. When surgical device 206 intersects target“T”, a virtual ablation zone 222 is projected onto image 218 as shown inFIG. 14B. The energy level and treatment duration can then be adjustedby a user and the controller 212 will adjust the virtual ablation zone222 to reflect the changes in the energy level and treatment duration.

The fiducial tracking system is described hereinbelow with reference toFIGS. 15-22. In the fiducial tracking system, controller 212 receives afiducial image from camera 208. Controller 212 also includes cameracalibration and distortion coefficients for camera 208, fiducial systemmodels, and camera-antenna calibration data previously stored thereon.In other embodiments, camera calibration and distortion coefficients forcamera 208, fiducial system models, and camera-antenna calibration datacan be entered into controller 212 during a navigation procedure. Basedon the fiducial image, camera calibration and distortion coefficientsfor camera 208, fiducial system models, and camera-antenna calibrationdata, controller 212 can output the position of ablation needle 206 todisplay 214 as well as diagnostic frame rate, residual error, andtracking status. In some embodiments, the distance between the camera208 and the fiducial patch 204 may be in the range of about 5 to about20 centimeters. In some embodiments, the distance between camera 208 andfiducial patch 204 may be in the range of about 1 to about 100centimeters.

FIG. 15 shows a basic flowchart for the fiducial tracking algorithmemployed by controller 212. As shown in FIG. 15, an image frame iscaptured in step 230. In step 231, controller 212 corrects for lensdistortion using the camera calibration and distortion coefficients.Images captured by camera 208 may exhibit lens distortion as shown inFIG. 16A. Thus, before an image can be used for further calculations,the image needs to be corrected for the distortion. Before camera 208 isused during a navigation procedure, camera 208 is used to take multipleimages of a checkerboard pattern at various angles. The multiple imagesand various angles are used to create a camera matrix and distortioncoefficients. Controller 212 then uses the camera matrix and distortioncoefficients to correct for lens distortion.

In step 232, controller 212 finds the white circles in the image frameusing the algorithm of FIG. 17. As shown in FIG. 17, the image framereceived in step 241 (FIG. 18A) is thresholded in step 243 using adynamic threshold (see FIG. 18B). When using a dynamic threshold, aftereach valid frame, the dynamic threshold algorithm computes a newthreshold for the next frame using the circles that were found in thevalid frame. Using the circles that were found in the valid frame,controller 212 calculates a new threshold based on equation (5) below:

threshold=(black circle intensity_(average)+white circleintensity_(average))/2  (5)

A predetermined threshold may be used to capture the initial valid framewhich is then used to calculate a new threshold.

Alternatively, controller 212 may scan for an initial threshold bytesting a range of threshold values until a threshold value is foundthat results in a valid frame. Once an initial threshold is found,controller 212 would use equation (5) for dynamic thresholding based onthe valid frame.

In other embodiments, a fixed threshold may be used. The fixed thresholdmay be a predetermined number stored in controller 212 or it may bedetermined by testing the range of threshold values until a thresholdvalue is found that results in a valid frame.

After a threshold and automatic gain control is applied to the image, aconnected component analysis is performed in step 244 to find all theobjects in the thresholded image. A geometric filter is applied to theresults of the connected component analysis and the image frame in step245. The geometric filter computes the size and shape of the objects andkeeps only those objects that are circular and about the right size asshown in FIG. 18C. Weighted centroids are computed and stored for allthe circular objects.

Turning back to FIG. 15, in addition to finding the white circles instep 232, controller 212 also finds the black circles in step 233 usingthe algorithm depicted in FIG. 19. The algorithm for finding the blackcircles is similar to the algorithm shown in FIG. 17 for finding thewhite circles. In order to find the black circles, after an image frameis received in step 241 (see FIG. 20A), controller 212 inverts theintensities of the image frame in step 242 as shown in FIG. 20B. Then,as described above with regard to FIG. 17, the image is thresholded asshown in FIG. 20C and the connected component analysis is performed andgeometric filter is applied to obtain the image shown in FIG. 20D. Theweighted centroids are computed and stored for all the black circles instep 248. Further, in step 245, controller 212 applies a geometricfilter to determine the black regions in addition to the black circlesin the image frame. Controller 212 stores the determined black regionsin step 249.

In step 234 of FIG. 15, controller 212 finds a correspondence betweenthe fiducial image and fiducial models using the algorithm of shown inFIG. 21A. In step 251 of FIG. 21A, controller 212 uses a topologyconstraint to select the four white circles as shown in FIG. 21B. Asshown in FIG. 21B, in step 261, controller 212 obtains the black regionsstored in step 249 of FIG. 19 and obtains the white circles stored instep 246 of FIG. 17. Controller 212 then selects a first black region instep 263 and counts the number of white circles in the first blackregion in step 264. Controller 212 determines whether the number ofcircles in the selected black region matches a predetermined number ofcircles in step 265. If the number of circles does not match thepredetermined number of circles, the algorithm proceeds to step 266where the next black region is selected and the number of circles in thenext black region is counted again in step 264. This process repeatsuntil the number of circles counted in step 264 matches thepredetermined number of circles. Once the number of circles counted instep 264 matches the predetermined number of circles, the algorithmproceeds to step 267 where the topology constraint algorithm iscompleted. In other embodiments, controller 212 selects the four whitecircles by selecting the four roundest circles.

After the four circles are chosen, they are arranged in a clockwiseorder using a convex hull algorithm in step 252. The convex hull orconvex envelope for a set of points X in a real vector space V is theminimal convex set containing X. If the points are all on a line, theconvex hull is the line segment joining the outermost two points. In theplanar case, the convex hull is a convex polygon unless all points areon the same line. Similarly, in three dimensions the convex hull is ingeneral the minimal convex polyhedron that contains all the points inthe set. In addition, the four matching fiducials in the model are alsoarranged in a clockwise order.

In step 253, a planar homography matrix is computed. After a planarhomography matrix is calculated, the homography matrix is used totransform the fiducial models to image coordinates using the fourcorresponding fiducial models shown in FIG. 22 to find the closestmatching image fiducials (steps 254 and 255). Controller 212 alsocomputes the residual error in step 256. The algorithm uses theresulting 3D transform to transform the 3D fiducial model into the 2Dimage. It then compares the distances between fiducials mapped into the2D image with the fiducials detected in the 2D image. The residual erroris the average distance in pixels. This error is used to verify accuracyand partly determine the red/green navigation status. Controller 212then selects the model with the most matches and the smallest residualerror. In order for a more accurate result, there has to be a minimumnumber of black fiducial matches (e.g., three).

In step 235 of FIG. 15, camera pose estimation is performed. The camerapose estimation involves calculating a 3D transform between the cameraand the selected model by iteratively transforming the model fiducialsonto the fiducial image plane and minimizing the residual error inpixels. The goal is to find the global minimum of the error function.One problem that may occur is the occurrence of significant local minima(e.g., an antenna imaged from the left looks similar to an antennaimaged from the right) in the error function that needs to be avoided.Controller 212 avoids the local minima by performing minimization frommultiple starting points and choosing the result with the smallesterror. Once the 3D transform is calculated, the controller can use the3D transform to transform the coordinates of the surgical device 206 toa model space and display the surgical device 206 as virtual surgicaldevice 206 a in display 214.

Because object boundaries expand and contract under different lightingconditions, a conventional square corner fiducials location may changedepending on lighting conditions. Fiducial patch 204 uses black andwhite circles, and, thus, is not hampered by this problem because thecenter of the circle always stays, the same and continues to work wellfor computing weighted centroids. Other contrasting images or colors arealso contemplated.

In another embodiment of the present disclosure, and as shown in FIG.23, a planning and navigation system 300 is provided. System 300includes planning system 302 and navigation system 304 that areconnected to a controller 306. Controller 306 is connected to a display308 that may include a single display screen or multiple display screens(e.g., two display screens). Planning system 302 is similar to planningsystem 100 and navigation system 304 is similar to navigation system200. In system 300, display 308 displays the planning operation andnavigation operation described hereinabove. The planning operation andthe navigation operation may be displayed as a split screen arrangementon a single display screen, the planning operation and the navigationoperation may be displayed on separate screens, or the planningoperation and the navigation operation may be displayed the same screenand a user may switch between views. Controller 306 may import dosesettings from the planning system and use the dose setting during anavigation operation to display the ablation zone dimensions.

In other embodiments of the present disclosure, CT navigation andsoftware can be integrated with planning system 100. Turning to FIGS.24, 25A, and 25B a planning and navigation system is shown generally as400. System 400 includes an image capturing device 402 that captures CTimages of a patient “P” having an electromagnetic reference 428 and/oroptical reference 438. The CT images are provided in DICOM format toplanning system 404 that is similar to planning system 100. Planningsystem 400 is used to determine a treatment plan as described above andthe treatment plan is provided to controller 408 and displayed as aplanning screen 412 on display 410 as shown in FIG. 26.

Navigation system 406 may use an electromagnetic tracking system asshown in FIG. 25A, an infrared tracking system or an optical trackingsystem as shown in FIG. 25B. Turning to FIG. 25A, a navigation system420 includes an electromagnetic field generator 422, an surgical device424 having an electromagnetic transducer 426, and an electromagneticreference 428 disposed on the patient. The field generator 422 emitselectromagnetic waves which are detected by electromagnetic sensors (notexplicitly shown) on the surgical device 424 and electromagneticreference 428 and then used to calculate the spatial relationshipsbetween surgical device 424 and electromagnetic reference 428. Thespatial relationships may be calculated by the field generator 422 orthe field generator 422 may provide the data to controller 408 tocalculate the spatial relationship between the ablation needle 424 andthe electromagnetic reference 428.

FIG. 25B depicts an alternate navigation system 430 that is similar tothe navigation system described in FIG. 13 above. In FIG. 25B, anoptical reference or fiducials 438 is placed on a patient. A camera 436attached to surgical device 424 takes an image of the fiducials 438 andtransmits the image to controller 408 to determine a position of theablation needle in relation to the fiducials 438.

After receiving data from navigation system 406, controller 408 maycorrelate the position of the surgical device 424 with the CT images inorder to navigate the surgical device 424 to a target “T” as describedbelow. In this case, the patient reference (of any type) may haveradiopaque markers on it as well to allow visualization during CT. Thisallows the controller to connect the patient CT image coordinate systemto the instrument tracking coordinate system.

Controller 408 and display 410 cooperate with each other to display theCT images on a navigation screen 440 as shown in FIG. 27. As shown inFIG. 27, display screen 440 includes a transverse view 442, coronal view444, and sagittal view 446. Each view includes a view of the target “T”and an ablation zone 452 (including a margin). The transverse view 442,coronal view 444 and sagittal view 446, ablation zone 452 are allimported from planning system 404. Additionally, all planning elements(e.g., device selection, energy level, and treatment duration) areautomatically transferred to the navigation screen 440. The navigationscreen 440 is also a graphical user interface that allows a user toadjust the device selection, energy level, and treatment duration.

A navigation guide screen 448 is provided on display screen 440 toassist in navigating the ablation needle to the target “T”. Based on thedata received from the navigation system 406, the controller candetermine if the surgical device 424 is aligned with target “T”. If thesurgical device 424 is not aligned with target “T”, the circle 454 wouldbe off-centered from outer circle 453. The user would then adjust theangle of entry for the surgical device 424 until the center of circle454 is aligned with the center of outer circle 453. In some embodiments,circle 454 may be displayed as a red circle when the center of circle454 is not aligned with the center of outer circle 453 or circle 454 maybe displayed as a green circle when the center of circle 454 is alignedwith the center of outer circle 453. Additionally, controller 408 maycalculate the distance between the target “T” and the surgical device424.

In another embodiment depicted in FIG. 28, controller 408 superimposes avirtual surgical device 424 a over a 3D rendered image and displays thecombined image on screen 462. Similar to the method described above, auser can align the center of circle 453 with the center of circle 454 tonavigate the surgical device 424 to the target “T”. Alternatively, theuser can determine the position of surgical device 424 in relation tothe target “T” by viewing virtual surgical device 424 a on screen 462 tonavigate the surgical device 424 to the target “T”.

FIG. 29 depicts another embodiment of the present disclosure. Similarlyto screen 462 above, in the embodiment of FIG. 29, screen 472 depicts avirtual surgical device 424 a in spatial relationship to previouslyacquired and rendered CT image. The CT image has been volume rendered todemarcate the target “T” as well as additional structures, vessels, andorgans. By volume rendering the target “T”, as well as the additionalstructures, vessels, and organs, the user can navigate the surgicaldevice 424 into the patient while also avoiding the additionalstructures, vessels, and organs to prevent unnecessary damage.

It should be understood that the foregoing description is onlyillustrative of the present disclosure. Various alternatives andmodifications can be devised by those skilled in the art withoutdeparting from the disclosure. Accordingly, the present disclosure isintended to embrace all such alternatives, modifications and variances.The embodiments described with reference to the attached drawing figuresare presented only to demonstrate certain examples of the disclosure.Other elements, steps, methods and techniques that are insubstantiallydifferent from those described above and/or in the appended claims arealso intended to be within the scope of the disclosure.

What is claimed is:
 1. A method of determining a treatment plan,comprising: obtaining a plurality of images; rendering the plurality ofimages in three dimensions; segmenting the plurality of images todemarcate a target area; and automatically determining a treatment planbased on the target area.
 2. The method of claim 1, whereinautomatically segmenting the plurality of images further comprises:selecting a seed point; creating a region of interest around the seedpoint; comparing a first plurality of pixels in the region of interestto a predetermined threshold; selecting a second plurality of pixelsfrom the first plurality of pixels, wherein the second plurality ofpixels are connected to the seed point and are less than thepredetermined threshold; and applying a geometric filter to the secondplurality of pixels.
 3. The method of claim 2, further comprising:determining if the second plurality of pixels forms a predeterminedobject, wherein if the second plurality of pixels does not form apredetermined object, the predetermined threshold is increased and thesteps of comparing a first plurality of pixels, selecting a secondplurality of pixels, applying a geometric filter, and determining if thesecond plurality of pixels forms a predetermined object are repeated. 4.The method of claim 1, wherein automatically determining a treatmentplan further comprises: performing a volumetric analysis on the targetarea; selecting a surgical device; and calculating an energy level andtreatment duration based on the target area and the selected surgicaldevice.
 5. The method of claim 1, further comprising: displaying therendered plurality of images; displaying the target area; and displayingthe treatment plan.
 6. The method of claim 1, further comprising:automatically segmenting at least one vessel; adjusting the treatmentplan based on a proximity of the at least one vessel to the target; anddisplaying the treatment plan.
 7. The method of claim 1, furthercomprising: automatically segmenting at least one organ; and adjustingthe treatment plan based on a location of the target in relation to theat least one organ; and displaying the treatment plan.
 8. A method ofnavigating a surgical device using a fiducial pattern disposed on anultrasound device and an image capture device disposed on the surgicaldevice, the method comprising: obtaining an ultrasound image of a scanplane; obtaining a fiducial image of the fiducial pattern; correctingthe fiducial image for lens distortion; finding correspondence betweenthe fiducial image and a model image; estimating a camera pose;transforming a position of the surgical device to model coordinates; anddisplaying the ultrasound image and a virtual image of the surgicaldevice based on the model coordinates.
 9. A method of tracking a firstdevice having an image capture device in relation to a second devicehaving a fiducial pattern, the method comprising: obtaining a fiducialimage of the fiducial pattern; correcting the fiducial image for lensdistortion; finding correspondence between the fiducial image and amodel image; estimating a camera pose; transforming a position of thesurgical device to model coordinates.
 10. The method of claim 9, whereinthe fiducial pattern includes a plurality of first unique identifiersdisposed in a region and a plurality of second unique identifiers. 11.The method of claim 10, further comprising finding the plurality offirst unique identifiers which includes: applying a first threshold tothe fiducial image; performing a connected component analysis; applyinga geometric filter to determine the weighted centroids of the pluralityof first unique identifiers; and storing the weighted centroids of theplurality of first unique identifiers.
 12. The method of claim 11,further comprising finding the plurality of second unique identifierswhich includes: inverting the fiducial image; applying a secondthreshold to the inverted fiducial image; performing a connectedcomponent analysis; applying a geometric filter to determine theweighted centroids of the plurality of second unique identifiers and todetermine the region having the plurality of first unique identifiers;and storing the weighted centroids of the plurality of secondidentifiers and the region having the plurality of first uniqueidentifiers.
 13. The method of claim 12, wherein finding correspondencebetween the fiducial image and the model image comprises: selecting aplurality of first unique identifiers from the fiducial image; arrangingthe plurality of first unique identifiers in clockwise order; arranginga plurality of model fiducials in clockwise order; computing a planarhomography; transforming the plurality of model fiducials into imagecoordinates using the computed planar homography; finding a modelfiducial from the plurality of model fiducials that matches the fiducialimage; and computing the residual error.
 14. The method of claim 13,wherein selecting a plurality of first unique identifiers from thefiducial image, comprises: selecting the plurality of first uniqueidentifiers; selecting the region having the plurality of first uniqueidentifiers; counting the number of first unique identifiers in theselected region; comparing the number of first unique identifiers in theselected region to a predetermined number, wherein if the number offirst unique identifiers in the selected region equals the predeterminednumber, the method proceeds to arranging the plurality of first uniqueidentifiers in clockwise order, and wherein if the number of firstunique identifiers in the selected region does not equal thepredetermined number, a new region is selected and the number of firstunique identifiers in the new region is counted.
 15. The method of claim12, wherein the first threshold and/or the second threshold are dynamicthresholds.
 16. A planning and navigation method, comprising: obtaininga plurality of images; rendering the plurality of images in threedimensions; automatically segmenting the plurality of images todemarcate a target area; automatically determining a treatment planbased on the target area; obtaining an ultrasound image of a scan planeincluding the target; obtaining a fiducial image of a fiducial patterndisposed on an ultrasound device using an image capture device on asurgical device; correcting the fiducial image for lens distortion;finding correspondence between the fiducial image and a model image;estimating a camera pose; transforming a position of the surgical deviceto model coordinates; displaying the ultrasound image and a virtualimage of the surgical device based on the model coordinates; navigatingthe surgical device to the target using the displayed ultrasound imageand the virtual image; and treating the target based on the treatmentplan.
 17. The method of claim 16, wherein automatically segmenting theplurality of images further comprises: selecting a seed point; creatinga region of interest around the seed point; comparing a first pluralityof pixels in the region of interest to a predetermined threshold;selecting a second plurality of pixels from the first plurality ofpixels, wherein the second plurality of pixels are connected to the seedpoint and are less than the predetermined threshold; and applying ageometric filter to the second plurality of pixels.
 18. The method ofclaim 17, further comprising: determining if the second plurality ofpixels forms a predetermined object, wherein if the second plurality ofpixels does not form a predetermined object, the predetermined thresholdis increased and the steps of comparing a first plurality of pixels,selecting a second plurality of pixels, applying a geometric filter, anddetermining if the second plurality of pixels forms the predeterminedobject are repeated.
 19. The method of claim 16, wherein automaticallydetermining a treatment plan further comprises: performing a volumetricanalysis on the target area; calculating an energy level and treatmentduration based on the target area and the surgical device.
 20. Themethod of claim 16, further comprising: displaying the renderedplurality of images; displaying the target area; and displaying thetreatment plan.
 21. The method of claim 16, further comprising:automatically segmenting at least one vessel; adjusting the treatmentplan based on a proximity of the at least one vessel to the target; anddisplaying the adjusted treatment plan.
 22. The method of claim 16,further comprising: automatically segmenting at least one organ;adjusting the treatment plan based on a location of the target inrelation to the at least one organ; and displaying the adjustedtreatment plan.
 23. The method of claim 16 wherein the fiducial patternincludes a plurality of first unique identifiers disposed in a regionand a plurality of second unique identifiers.
 24. The method of claim23, wherein finding the plurality of first unique identifiers comprises:applying a first threshold to the fiducial image; performing a connectedcomponent analysis; applying a geometric filter to determine theweighted centroids of the plurality of first unique identifiers; andstoring the weighted centroids of the plurality of first uniqueidentifiers.
 25. The method of claim 24, wherein finding the pluralityof second unique identifiers comprises: inverting the fiducial imageapplying a second threshold to the inverted fiducial image; performing aconnected component analysis; applying a geometric filter to determinethe weighted centroids of the plurality of second unique identifiers andto determine the region having the plurality of first uniqueidentifiers; and storing the weighted centroids of the plurality ofsecond identifiers and the region having the plurality of first uniqueidentifiers.
 26. The method of claim 25, wherein finding correspondencebetween the fiducial image and the model image comprises: selecting aplurality of first unique identifiers from the fiducial image; arrangingthe plurality of first unique identifiers in clockwise order; arranginga plurality of model fiducials in clockwise order; computing a planarhomography; transforming the plurality of model fiducials into imagecoordinates using the computed planar homography; finding a modelfiducial from the plurality of model fiducials that matches the fiducialimage; and computing the residual error.
 27. The method of claim 26,wherein selecting a plurality of first unique identifiers from thefiducial image, comprises: selecting the plurality of first uniqueidentifiers; selecting the region having the plurality of first uniqueidentifiers; counting the number of first unique identifiers in theselected region; comparing the number of first unique identifiers in theselected region to a predetermined number, wherein if the number offirst unique identifiers in the selected region equals the predeterminednumber, the method proceeds to arranging the plurality of first uniqueidentifiers in clockwise order, and wherein if the number of firstunique identifiers in the selected region does not equal thepredetermined number, a new region is selected and the number of firstunique identifiers in the new region is counted.
 28. The method of claim25, wherein the first threshold and/or the second threshold are dynamicthresholds.